In my current organization, we use the Kirkpatrick Model as our guiding framework for evaluation. This model looks at training impact across four key levels: reaction, learning, behavior, and results. I've found it to be a comprehensive and effective way to understand the full scope of our program's effectiveness. So, starting with that first level - reaction - we always make sure to gather feedback from participants immediately after a training session. We use a mix of quantitative and qualitative methods here. Every learner fills out a post-training survey where they rate various aspects of the experience on a scale, things as the relevance of the content, the effectiveness of the instructor, the engagement level of the activities, and so on.

But we also make sure to include open-ended questions where they can share more detailed thoughts and suggestions. And we complement this with focus groups and one-on-one conversations to dive deep into their experience. I remember one training we did on a new software system, and the survey feedback was mostly positive. However, when we dug into the qualitative comments, we realized there was a common frustration with the hands-on practice sessions. Learners felt they needed more time to get comfortable with the tool. That was invaluable insight that we used to adjust the pacing and structure of future sessions.

Moving on to level two - learning - this is where we assess how well participants have absorbed and retained the knowledge or skills from the training. The specific methods vary depending on the content but often include things like pre- and post-training quizzes, case study analyses, or skill demonstrations. For example, in a recent training on consultative selling techniques, we had participants role-play various sales scenarios before and after the training. We video-recorded these so we could see the difference in their approach and technique. The progress was remarkable - after the training, their questioning was more strategic, their positioning was more benefit-focused, and their closing was more confident. That kind of tangible, observable learning gain is so powerful to see.

But of course, the real test is whether that learning translates into on-the-job behavior change. That's level three of the Kirkpatrick model, and it's where the rubber meets the road in terms of impact. For this, we rely heavily on partnerships with managers and ongoing reinforcement and coaching. About a month after training, we always send out a follow-up survey to participants and their managers to understand how they're applying the learning in their day-to-day work. We ask about specific behaviors they were supposed to adopt, any challenges they're facing, additional support they need, and so on.

We also work with managers to set up opportunities for ongoing practice and feedback. So for that consultative selling training, we had managers do ride-alongs with their reps and provide coaching based on the techniques they learned. We also set up a peer mentoring program where more experienced reps could guide and support their colleagues. By really embedding the learning into the flow of work like this, we dramatically increase the chances of sustained behavior change.

Finally, the holy grail of training evaluation is level four - results. This is where we look at the business impact of our programs. And it's admittedly the most challenging to measure, but also the most crucial for demonstrating the strategic value of what we do. Wherever possible, we try to tie our training initiatives directly to key performance indicators. So for that consultative selling program, we looked at metrics like conversion rates, average deal size, and customer satisfaction scores. The results were impressive - within three months of the training, we saw a 15% increase in conversion, a 20% increase in average deal size, and a 10-point boost in customer satisfaction. Being able to quantify our impact like that is so powerful in terms of gaining buy-in and investment from leadership. Of course, not every training will have such a clear and immediate link to business results. For some, the impact is more indirect or long-term. But we still strive to find ways to connect the dots and tell that impact story.

For a leadership development program we ran last year, for instance, we knew the ultimate goal was to strengthen our succession pipeline and retain high-potential talent. So we tracked metrics like internal promotion rates and retention of program participants over time. We also collected qualitative feedback from senior leaders on the readiness and capabilities of the cohort. While it wasn't as cut-and-dried as sales figures, this combination of quantitative and qualitative data painted a compelling picture of the program's impact.

Across all of these levels and methods, the key is to be rigorous, consistent, and always focused on continuous improvement. Every piece of evaluation data we collect is an opportunity to learn and adjust our approach. We regularly review our findings as a team, identify trends and insights, and use them to inform our strategy going forward.

Written by William Rosser on March 12th, 2024